Automated asset management system with multiple sensing technologies

ABSTRACT

An automated asset management system includes a plurality of storage locations for storing objects, and first and second sensing subsystems each configured to sense presence or absence of the objects in the plurality of storage locations. The first and second sensing subsystems are used to sense the presence or absence of a same particular object using different respective sensing modalities. In operation, a first scan of the storage locations is performed using the first sensing subsystem, and the presence or absence of the particular object is determined using the first sensing modality. In turn, a second scan of the storage locations is performed using the second sensing subsystem, and the presence or absence of the particular object is confirmed using both the result of the determination made using the first sensing modality and a determination of the presence or absence of the particular object using the second sensing modality.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation of U.S. patent application Ser. No.15/681,964, filed on Aug. 21, 2017, which is a continuation of U.S.patent application Ser. No. 15/097,802, filed on Apr. 13, 2016, now U.S.Pat. No. 9,741,014, which claims the benefit of U.S. Provisional PatentApplication No. 62/147,891, filed in the U.S. Patent and TrademarkOffice on Apr. 15, 2015, the disclosures of each are incorporated byreference herein in their entirety.

TECHNICAL FIELD

The present subject matter relates to automated tool control systems,and to techniques and equipment to automatically track tools stored inautomated tool control systems using multiple sensing technologies.

BACKGROUND

When tools are used in a manufacturing or service environment, it isimportant that tools be returned to a storage unit, such as a tool box,after use. Some industries have high standards for inventory control oftools, for example to prevent incidents of leaving tools in theworkplace environment where they could cause severe damages. In theaerospace industry, for instance, it is important to ensure that notools are accidentally left behind in an aircraft or missile beingmanufactured, assembled, or repaired in order to prevent foreign objectdamage (FOD) to the aircraft.

Some toolboxes include built-in inventory determination features totrack inventory conditions of tools stored in those toolboxes. Forexample, some toolboxes dispose contact sensors, magnetic sensors, orinfrared sensors in or next to each tool storage location to detectwhether a tool is placed in the tool storage location. Based on signalsgenerated by the sensors, the toolboxes are able to determine whetherany tool is missing.

The different types of sensors used in toolboxes each have distinctadvantages and disadvantages, and different associated costs. Forexample, certain sensors may provide real-time or near-instantaneousinformation on the status of a tool upon the tool being placed in thetoolbox, while other sensors may have associated delays. Certain sensorsmay not differentiate between a tool and another object having a similarweight, shape, or other sensed attribute, and may therefore notdifferentiate between the tool and the other object being present in thetoolbox. Other sensors may not differentiate between multiple similartools, and may therefore not be able to determine whether a toolreturned by one user into the toolbox was the same tool borrowed by theuser or another similar tool borrowed by another user.

A need therefore exists for automated asset management systems thatleverage the advantages of multiple different sensing technologieswithin a same system—and with regard to a same tool—to provide moreprecise and more efficient automated asset management.

SUMMARY

The teachings herein improve the efficiency and tracking capability ofasset management systems to automatically track objects stored thereinby concurrently using multiple sensing technologies.

In accordance with one aspect of the disclosure, an automated assetmanagement system includes a plurality of storage locations for storingobjects, first and second sensing subsystems each configured to sensepresence or absence of the objects in the plurality of storage locationsof the asset management system, a processor, and a non-transitorymachine readable recording medium storing program instructions. Thefirst and second sensing subsystems are configured to sense the presenceor absence of a same particular object in the asset management systemusing different respective first and second sensing modalities. Theprogram instructions, when executed by the processor, cause theprocessor to perform a scan of the storage locations using the firstsensing subsystem, and determine the presence or absence of theparticular object in the plurality of storage locations using the firstsensing modality of the first sensing subsystem. The programinstructions further cause the processor to perform a scan of thestorage locations using the second sensing subsystem, and confirm thepresence or absence of the particular object in the plurality of storagelocations using both the result of the determination made using thefirst sensing modality and a determination of the presence or absence ofthe particular object using the second sensing modality of the secondsending subsystem.

The second sensing subsystem may be further configured to read uniqueidentifiers from objects present in the asset management system. In oneexample, the second sensing subsystem may be configured to read uniqueidentifiers stored in radio frequency (RF) identification (RFID) tagsassociated with objects present in the asset management system. Inanother example, the second sensing subsystem may be configured to readunique identifiers stored in bar-code or quick response (QR) code tagsassociated with objects present in the asset management system. In theother example, the first sensing subsystem may be a camera-based imagesensing subsystem including one or more cameras, the second sensingsubsystem may be a tag sensing subsystem for sensing bar-code or quickresponse (QR) code tags associated with objects in the asset managementsystem, and the first and second sensing subsystems may use the same oneor more cameras for performing scans of the storage locations using thefirst and second sensing modalities.

The first sensing subsystem may be a camera-based image sensingsubsystem including one or more cameras, and may be configured todetermine the presence or absence of a particular tool using acamera-based visual sensing modality by capturing an image of a storagelocation associated with the particular tool and determining whether thetool is present in the captured image. The second sensing subsystem maybe a radio frequency (RF)-based sensing subsystem including one or moreRF identification (RFID) transceivers, and may be configured todetermine the presence or absence of the particular tool using anRF-based wireless sensing modality used to sense whether an RF-based tagassociated with the particular tool is present in the asset managementsystem. The processor may be configured to determine a total number ofRF-based tags that are expected to be present in the asset managementsystem using the first sensing modality, and perform the scan of thestorage locations using the RF-based sensing subsystem to identify thedetermined total number of RF-based tags in the asset management system.The RF-based sensing subsystem may be used to confirm the presence orabsence of the particular object using both the result of thedetermination made by the camera-based visual sensing modality and theRF-based wireless sensing modality.

In accordance with a further aspect of the disclosure, a method includesperforming, using a first sensing subsystem, a first scan of a pluralityof storage locations for storing objects in an automated assetmanagement system having the first sensing subsystem and a secondsensing subsystem each configured to sense presence or absence of theobjects in the plurality of storage locations. Presence or absence of aparticular object in the plurality of storage locations is determinedbased on a result of the first scan using a first sensing modality ofthe first sensing subsystem. A second scan of the plurality of storagelocations is performed using the second sensing subsystem. In turn, thepresence or absence of the particular object in the plurality of storagelocations is confirmed using both a result of the determination madeusing the first sensing modality and a determination of the presence orabsence of the particular object using a second sensing modality of thesecond sending subsystem. The first and second sensing subsystems areconfigured to sense the presence or absence of a same particular objectin the asset management system using different respective first andsecond sensing modalities.

The performing of the second scan can include reading, using the secondsensing subsystem, unique identifiers from objects present in the assetmanagement system. In one example, the performing of the second scan caninclude reading, using the second sensing subsystem, unique identifiersstored in radio frequency (RF) identification (RFID) tags associatedwith objects present in the asset management system. In another example,the performing of the second scan can include reading, using the secondsensing subsystem, unique identifiers stored in bar-code or quickresponse (QR) code tags associated with objects present in the assetmanagement system. In the other example, the first sensing subsystem maybe a camera-based image sensing subsystem including one or more cameras,the second sensing subsystem may be a tag sensing subsystem for sensingbar-code or quick response (QR) code tags associated with objects in theasset management system, and the first and second sensing subsystems mayuse the same one or more cameras for performing the first and secondscans of the storage locations using the first and second sensingmodalities.

The first sensing subsystem may be a camera-based image sensingsubsystem including one or more cameras, the performing the first scanmay include capturing an image of a storage location associated with theparticular object, and the determining the presence or absence of theparticular object based on the result of the first scan may includedetermining whether the object is present in the captured image. Thesecond sensing subsystem may be a radio frequency (RF)-based sensingsubsystem including one or more RF identification (RFID) transceivers,the performing the second scan may include sensing whether an RF-basedtag associated with the particular object is present in the assetmanagement system, and confirming the presence or absence of theparticular object is based on the sensing whether the RF-based tagassociated with the particular object is present in the asset managementsystem. The method may further include determining a total number ofRF-based tags that are expected to be present in the asset managementsystem based on the result of the first scan, where the performing ofthe second scan of the storage locations includes using the RF-basedsensing subsystem to identify the determined total number of RF-basedtags in the asset management system. The confirming the presence orabsence of the particular object may include using both the result ofthe determination made by a camera-based visual sensing modality and aRF-based wireless sensing modality.

Additional advantages and novel features will be set forth in part inthe description which follows, and in part will become apparent to thoseskilled in the art upon examination of the following and theaccompanying drawings or may be learned by production or operation ofthe examples. The advantages of the present teachings may be realizedand attained by practice or use of various aspects of the methodologies,instrumentalities and combinations set forth in the detailed examplesdiscussed below.

BRIEF DESCRIPTION OF THE DRAWINGS

The drawing figures depict one or more implementations in accord withthe present teachings, by way of example only, not by way of limitation.In the figures, like reference numerals refer to the same or similarelements.

FIGS. 1A-1C show various illustrative automated asset management systemsin the form of tool storage systems.

FIGS. 2A and 2B are high-level functional block diagrams of an automatedasset management system and of sensing subsystems thereof.

FIG. 3 shows an opened drawer of an automated asset management systemlike those shown in FIGS. 1A-1C.

FIG. 4 shows components of an image sensing subsystem in an illustrativeautomated asset management system like that shown in FIG. 1C.

FIG. 5 is a functional flow diagram showing steps of a method forperforming an inventory of objects using multiple sensing technologiesin an automated asset management system.

DETAILED DESCRIPTION

In the following detailed description, numerous specific details are setforth by way of examples in order to provide a thorough understanding ofthe relevant teachings. However, it should be apparent to those skilledin the art that the present teachings may be practiced without suchdetails. In other instances, well known methods, procedures, components,and/or circuitry have been described at a relatively high-level, withoutdetail, in order to avoid unnecessarily obscuring aspects of the presentteachings.

The various systems and methods disclosed herein relate to the use ofmultiple sensing technologies within an automated asset managementsystem. More particularly, the various systems and methods disclosedherein relate to the use of several different sensing technologies in anautomated asset management system to independently sense the presence(and/or other attributes) of a particular tool (or multiple particulartools)—and to thereby provide improved efficiency and accuracy, trackadditional information, and provide additional functionality to theautomated asset management system.

For example, an automated asset management system such as a toolbox mayuse both camera-based and radio-frequency (RF) based sensingtechnologies to sense the presence and/or other attributes of aparticular tool (or of multiple tools). The camera-based sensing mayprovide an instantaneous (or near-instantaneous) indication of whetherthe particular tool is present in or absent from the system. TheRF-based sensing may enable the system to differentiate between multipletools that are identical (or not differentiated) to the camera-basedsensing (e.g., similar torque wrenches), for example by distinguishingbetween the tools' serial numbers (or other unique identifiers) or otherunique tool identifiers encoded in a RF-based tag. Further, theautomated asset management system may be configured to more efficientlyperform RF-based sensing by leveraging the combined use of thecamera-based and RF-based sensing modalities as described in more detailbelow.

In another example, the automated asset management system may use bothimage-based and bar-code based sensing to sense the presence and/orother attributes of tools. As in the example described in the previousparagraph, the camera-based sensing may provide an indication of whetherparticular tools are present in or absent from the system. The bar-codebased sensing may enable the system to differentiate between tools thatare indistinguishable using the image-based analysis, for example bydistinguishing between serial numbers or other unique identifiersencoded in bar-codes affixed to the tools. The image-based and bar-codebased sensing may be performed using a same camera (or multi-camerasystem), or using distinct sensors (e.g., a first set of camera sensorsused for image-based sensing, and a second set of bar-code scanners usedfor bar-code sensing).

Reference now is made in detail to the examples illustrated in theaccompanying drawings and discussed below.

FIGS. 1A-1C show various illustrative automated asset management systems(or inventory control systems) in the form of a tool storage system 300.While the tool storage systems 300 shown in FIGS. 1A-1C are toolboxes,the tool storage systems 300 may more generally be tool lockers or anyother secure storage devices or enclosed secure storage areas (e.g., atool crib or walk-in tool locker).

Each tool storage system 300 is an example of a highly automatedinventory control system that utilizes multiple different sensingtechnologies for identifying inventory conditions of objects in thestorage unit. In one example, the tool storage system 300 uses machineimaging and RF sensing methodologies for identifying inventoryconditions of objects in the storage unit.

Illustrative features include the ability to process complex image datawith efficient utilization of system resources, autonomous image andcamera calibrations, identification of characteristics of tools fromimage data, adaptive timing for capturing inventory images, efficientgeneration of reference data for checking inventory status, autonomouscompensation of image quality, etc. Further features include the abilityto emit and receive RF sensing signals such as RF identification (RFID)signals, to process the received signals to identify particular tools,and to cross-reference tool information obtained through the multipledifferent sensing modalities (e.g., camera and RFID based modalities) toprovide advanced features. More detailed information on the tool storagesystem 300 can be found in U.S. application Ser. No. 12/484,127,entitled IMAGE-BASED INVENTORY CONTROL SYSTEM AND METHOD and filed onJun. 12, 2009, now patented as issued as U.S. Pat. No. 9,041,508 issuedMay 26, 2015, which is hereby incorporated by reference in its entirety.

As shown in each of FIGS. 1A-1C, the tool storage system 300 includes auser interface 305, an access control device 306, such as a card reader,for verifying identity and authorization levels of a user intending toaccess storage system 300, and multiple tool storage drawers 330 forstoring tools. Instead of drawers 330, the storage system may includeshelves, compartments, trays, containers, or other object storagedevices from which tools or objects are issued and/or returned, or whichcontain the storage device from which the objects are issued and/orreturned. In further examples, the storage system includes storagehooks, hangers, tool boxes with drawers, lockers, cabinets with shelvesand/or doors, safes, boxes, closets, vending machines, barrels, crates,and other material storage means.

User interface 305 is an input and/or output device of storage system330, configured to display information to a user. Access control device306 is used to limit or allow access to tool storage drawers 330. Accesscontrol device 306, through the use of one or more electronicallycontrolled locking devices or mechanisms, keeps some or all storagedrawers 330 locked in a closed position until access control device 306authenticates a user's authorization for accessing storage system 300.The access control device 306 further includes a processor and softwareto electronically identify a user requesting access to the secure areaor object storage device and determine the level of access which shouldbe granted or denied to the identified user. If access control device306 determines that a user is authorized to access storage system 300,it unlocks some or all storage drawers 330, depending on the user'sauthorization level, allowing the user to remove or replace tools. Inparticular, the access control device 306 may identify predeterminedauthorized access levels to the system (e.g., a full access levelproviding access to all drawers 330, a partial access level providingaccess only to particular drawer(s) 330, or the like), and allow or denyphysical access by the user to the three dimensional space or objectstorage devices based on those predetermined authorized levels ofaccess.

Tool storage system 300 includes several different sensing subsystems.In an illustrative example, the tool storage system 300 includes a firstsensing subsystem in the form of an image sensing subsystem configuredto capture images of contents or storage locations of the system. Theimage sensing subsystem may include one or more lens-based cameras, CCDcameras, CMOS cameras, video cameras, or other types of devices thatcaptures images. The tool storage system 300 further includes a secondsensing subsystem that, in one example, takes the form of an RFIDsensing subsystem including one or more RFID antennas, RFIDtransceivers, and RFID processors. The RFID sensing subsystem isconfigured to emit RF sensing signals when an RF-based scan of thestorage system 300 is performed, receive RFID signals returned from RFIDtags mounted on or incorporated in tools or other inventory items inresponse to the emitting the RF sensing signals, and process thereceived RFID signals to identify individual tools or inventory items.

The image sensing subsystem is described in further detail below inrelation to FIG. 3. While FIG. 3 corresponds to the specific embodimentof the storage system 300 shown in FIG. 1C, the teachings illustrated inFIG. 3 can be applied to each of the embodiments of FIGS. 1A-1C and toother types of automated asset management systems. The RFID sensingsubsystem may be configured to sense RFID tags of tools located in allstorage drawers 330 of system 300, or configured to sense RFID tags oftools located in a particular subset of the drawers 330 of system 300.In one example, the RFID sensing subsystem is configured to sense RFIDtags of tools located only in the top-most and bottom-most drawers 330of system 300, and the RFID sensing subsystem includes RFID antennasdisposed directly above the top-most and bottom-most drawers 330 withinsystem 300 to sense RFID tags of tools located in those drawers. Otherconfigurations of RFID antennas can also be used.

FIG. 2A is a block diagram showing components of an automated assetmanagement system 100, such as tool storage system 300. As shown in FIG.2A, and similarly to tool storage system 300, automated asset managementsystem 100 includes storage locations 130, a user interface 105, anaccess control device 106, and a network interface 108. The storagelocations 130 may include one or more storage drawers 330, shelves,cabinet doors, or the like. The user interface 105 may include one ormore user input/output devices, such as a display (e.g., atouch-sensitive display), a keyboard, a mouse or touchpad, a speakerand/or microphone, or the like. The access control device 106 mayinclude one or more of a card reader (e.g., identification card reader),an iris scanner, a locking mechanism, an alarm, or the like. The networkinterface 108 enables the system 100 to communicate across one or morewired or wireless networks with other network automated asset managementsystems, other tool storage systems (e.g., 300), or an asset managementserver that may be used to monitor the operation of and inventory statusof multiple tool storage systems.

Automated asset management system 100 further includes a data processingsystem 140, such as a computer, for processing sensing data receivedfrom various sensing subsystems 150 a and 150 b (reference genericallyas sensing subsystem(s) 150) and determining inventory conditions basedon the sensing data. In one example, the data processing system 140processes images captured by an image sensing device of the sensingsubsystem 150, for processing RFID signals captured by the RFID antennasand transceivers of the sensing subsystem 150, and/or for processingother sensing signals received by other sensing subsystems 150. The dataprocessing system 140 includes one or more processors 142 (e.g.,micro-processors) and memory 144. The memory 144 includes a programmemory storing program instructions for causing the automated assetmanagement system 100 to perform inventory control functions. The memory144 includes a database of tool information, which may include toolidentifiers, tool images, tool tag information (e.g., for RFID orbar-code tags), tool inventory status, and the like. The programinstructions further cause the system 100 to communicate electronicallydirectly or through a network with sensing devices (e.g., 150) andobtain data from sensing devices relative to the presence or absencedata of objects within the three dimensional space or object storagedevice. Images, RFID signals, and other sensing signals captured orreceived by the sensing subsystems 150 are processed by the dataprocessing system 140 for determining an inventory condition of thesystem 100 and/or of each storage drawer (e.g., 130).

The system 100 includes two or more sensing subsystems 150 a and 150 b.Each sensing subsystem relies on one or more sensor(s) to determine thepresence or absence of objects in the system 100. In one example, afirst sensing subsystem (e.g., 150 a) includes one or more cameras (orother image sensors), while a second sensing subsystem (e.g., 150 b)includes one or more RFID transceivers (or other RF sensors). In anotherexample, a first image-based sensing subsystem and a secondbar-code-based sensing subsystem use respective sensors (e.g., camera(s)and bar-code scanners, respectively). In a further example, the firstimage-based sensing subsystem and the second bar-code-based sensingsubsystem use same sensors (e.g., one or more camera(s)). In eachexample, the data processing system 140 processes sensing data obtainedfrom the sensor(s) in order to determine inventory conditions of theautomated asset management system 100.

The components of the automated asset management system 100 of FIG. 2Aare communicatively connected with each other, for example via acommunication bus or other communication links. The data processingsystem 140 functions as a central processing unit (CPU) for executingprogram instructions, such as program instructions stored in anon-transitory machine readable storage medium (e.g., memory 144), forcontrolling the operation of the asset management system 100.Additionally, each of the sensing subsystems (e.g., 150) can includemicroprocessors operative to execute program instructions and performfunctions relating to sensing operations. The automated asset managementsystem 100 can also be in communication, via network interface 108, withwired and/or wireless local area and/or wide area networks (e.g., theInternet). The automated asset management system 100 may communicatewith other asset management systems and/or servers across thenetwork(s), and may exchange information on inventory conditions, storedobjects, and operation data with those systems and/or servers.

Various examples of sensing subsystems are shown in FIG. 2B. Forexample, an image-sensing subsystem may include one or more imagesensor(s), such as lens-based cameras, CCD cameras, CMOS cameras, videocameras, or other types of devices that captures images. In examplesincluding multiple cameras, the cameras may have different fields ofview that may overlap with each other at the margins. For example,different cameras may have fields of view covering different drawers330, and/or fields of view covering different portions of a same drawer330. In operation, the image-sensing subsystem may rely on images of thedrawers 330 and/or objects stored in memory 144 to determine inventoryconditions.

A bar-code sensing subsystem may include one or more bar-code sensor(s),such as sensors for scanning uni-dimensional (1D) bar-codes,multi-dimensional (e.g., 2D) bar-codes, and/or quick response (QR)codes. The bar-code sensor(s) can be image sensor(s) (e.g., the sameimage sensor(s) used by an image-sensing subsystem used in the system100), bar-code scanners (e.g., a bar-code scanner that emits light), orthe like. In some examples, multiple bar-code scanners are used, forexample different bar-code scanners for different drawers 330, differentbar-code scanners for scanning different portions of a drawer 330, orthe like. In operation, the bar-code sensing subsystem may rely on adatabase stored in memory 144 that associates bar-codes with objects todetermine inventory conditions.

An RF sensing subsystem may include one or more RFID antenna(s) and RFIDtransceiver(s). RFID antenna(s) (and transceiver(s)) may be located atvarious locations within the system 100 in order to detect RFID tagswithin each antenna's vicinity. For example, an RF sensing subsystem mayinclude one or more RFID antennas located in each drawer 330 orpositioned to be directly above each drawer 330 when the drawer isclosed. The RF sensing subsystem may include RFID antennas andtransceivers only in (or proximate to) drawers 330 that are configuredto store objects equipped with RFID tags, such as only in (or proximateto) an uppermost and a lowermost drawer 330 of a tool storage system300. In operation, the RF sensing subsystem may rely on a databasestored in memory 144 that associates RFID tag numbers with objects todetermine inventory conditions.

The term inventory condition as used throughout this disclosure meansinformation relating to an existence/presence or non-existence/absencecondition of objects in the storage system.

The data processing system 140 may be part of and located within a toolstorage system 300. Alternatively, the data processing system 140 can bea remote computer having a data link, such as a wired or wireless link,coupled to tool storage system 300, or a combination of a computerintegrated in storage system 300 and a computer remote from storagesystem 300. Additionally, the data processing system 140 can beconnected to a computer network and exchange data with an administrativesoftware application (e.g., as may be executed on a server) used tomanipulate and store data and store and display information relative tothe data to system users.

FIG. 3 shows a detailed view of one illustrative drawer 330 of thestorage system 300 in an open position. The storage drawer 330 includesa foam base 180 having a plurality of storage locations, such as toolcutouts 181, for storing tools. Each cutout is specifically contouredand shaped for fittingly receiving a tool with a corresponding shape.Tools may be secured in each storage location by using hooks, Velcro,latches, pressure from the foam, etc.

In general, each storage drawer 330 includes multiple storage locationsfor storing various types of tools. As used throughout this disclosure,a storage location is a location in a storage system for storing orsecuring objects. In some embodiments, each tool has a specificpre-designated storage location in the tool storage system. In otherembodiments, multiple storage locations may have similar (or identical)shapes, and several similarly shaped tools may thus be placed in any ofthe multiple storage locations.

As shown in FIG. 3, one or more tools in the drawer 330 may haveidentification tags 331 a and 331 b mounted or attached thereon ortherein. The identification tags may be RFID tags, bar-code tags, or thelike. In the case of RFID tags, the RFID tags may be placed on a surfaceof the tools and may thus be visible to users, such as tag 331 a, or theRFID tags may be placed within the tool or may otherwise not be visibleto users, such as tag 331 b. In general, bar-code tags would be placedon a surface of the tools, such as tag 331 a.

While only some tools are shown in FIG. 3 as having identification tagsmounted thereon, in some embodiments all tools in a drawer 330 will beequipped with identification tags. Furthermore, all tools may beequipped with visible tags, invisible tags, or a combination thereof.

As described above, the automated asset management system 100 includestwo or more sensing subsystems 150. Various examples of sensingsubsystems 150 are described in relation to the following figures.

FIG. 4 shows a perspective view of an image-based sensing subsystem intool storage system 300. As illustrated in FIG. 4, storage system 300includes an imaging compartment 315 which houses an image sensingsubsystem comprising three cameras 310 and a light directing device,such as a mirror 312 having a reflection surface disposed at about 45degrees downwardly relative to a vertical surface, for directing lightreflected from drawers 330 to cameras 310. The directed light, afterarriving at cameras 310, allows cameras 310 to form images of drawers330. The shaded area 340 below mirror 312 represents a viewing field ofthe imaging sensing subsystem of tool storage system 300. As shown at340, the imaging subsystem scans a portion of an open drawer 336 thatpasses through the field of view of the imaging sensing subsystem, forexample as the drawer 336 is opened and/or closed. The imaging subsystemthereby captures an image of at least that portion of the drawer 336that was opened. Processing of the captured image is used to determinethe inventory conditions of tools and/or storage locations in theportion of the drawer 336 that was opened.

While the particular set of cameras 310 and mirror 312 shown in FIG. 4are configured to capture images of the drawers 330 by scanning thedrawers 330 as they are opened and/or closed, other tool storage systems300 may include additional or alternative set-ups enabling images of thedrawers 330 to be captured when the drawers are opened, closed, or inintermediate positions. For example, systems includes movable mirrors312, movable cameras 310, wide-angle cameras, concave or convex mirrors,or the like may be used to enable images of the drawers 330 to becaptured when the drawers are closed.

In general, the image sensing subsystem captures an image of aparticular drawer 330 and performs an inventory of the drawer inresponse to detecting movement of the particular drawer. For example,the image sensing subsystem may perform an inventory of the drawer inresponse to detecting that the drawer is closing or has becomecompletely closed. In other examples, the image sensing subsystem mayimage the drawer both as it is opening and as it closes.

The RF sensing subsystem is generally configured to perform inventorychecks of drawers having RF-based tags associated therewith. TheRF-based tags may be RFID tags that are attached to or embedded withinthe tools. In general, the RF-based tag encodes an identifier unique tothe tool, such that both the tool type (e.g., screwdriver, torquewrench, or the like) and the unique tool (e.g., a particular torquewrench, from among a plurality of torque wrenches of the model and type)can be identified from reading the RF-based tag. In particular, theinformation encoded in the RF-based tag is generally unique to the toolsuch that it can be used to distinguish between two tools that are of asame type, same model, same age, same physical appearance, etc.

The RF sensing system includes antennas mounted in or around the toolstorage system 300. In general, the antennas may be mounted inside thetool storage system 300 and be configured to only detect the presence ofRF-based tags that are located within the tool storage system 300 (orother defined three dimensional space). In some examples, each antennamay be mounted so as to only detect the presence of RF-based tags thatare located within a particular drawer or compartment of the toolstorage system 300, and different antennas may be associated with andmounted in different drawers or compartments. In further embodiments,some antennas may further be configured to detect the presence ofRF-based tags in the vicinity of the tool storage system 300 even if thetags are not located within the system 300.

Each antenna is coupled to an RF transceiver that is operative to causethe antenna to emit an RF sensing signal used to excite the RF-basedtags located within the vicinity of the antenna, and is operative tosense RF identification signals returned by the RF-based tags inresponse to emitting the RF sensing signal. One or more RF processorscontrol the operation of the RF transceivers and process the RFidentification signals received through the antennas and transceivers.

In general, the RF sensing subsystem performs an RF-based scan of thetool storage system 300 when a drawer or compartment storing toolshaving RF identification tags is completely closed. In particular, theRF-based scan can be performed in response to detecting that the drawerhas been completely closed, or performed at any time when the drawer iscompletely closed. In some examples, the RF-based scan can also betriggered by a user logging into or logging out of the tool storagesystem 300. In general, an RF-based scan can be performed in response tosimilar triggers causing a camera-based inventory of the tool storagesystem 300 to be performed.

As part of performing an RF-based scan of the tool storage system 300,the RF processor typically needs to perform multiple sequential scans inorder to ensure that all RF-based tags are detected. Specifically, theRF processor generally does not know how many RF tags it needs todetect, since one or more tags may be missing (e.g., if a tool has beenchecked out). Further, the RF processor cannot generally ensure that allRF tags in its vicinity have been detected in response to a single scanoperation (corresponding to the emission of one RF sensing signal, andthe processing of any RF identification responses received in responseto the one RF sensing signal). As a result, the RF processor willgenerally perform ten, twenty, or more sequential RF-based scans anytime an inventory of the tool storage system 300 is to be performed.Because multiple RF-based scans need to be performed, the RF scanningoperation may require 10 or more seconds to be performed, resulting insignificant inconvenience to users of the tool storage system 300.

As noted above, imaging-based inventory scans of the tool storage system300 have the disadvantage that they cannot distinguish betweenphysically identical tools. Further, RF-based scans of the tool storagesystem 300 may suffer from significant delay, and cannot determine if anRF tag alone (instead of an RF tag attached to its associated tool) hasbeen returned to the drawer or storage compartment. Both scanningmethodologies, when used alone, are thus susceptible to fraud (by usingtool cut-out, or using RFID tag removed from tool) and inconvenience.Further, each technology may not be suitable for inventorying all toolsin a particular system 300; for example, some tools may be too small tohave an RF-based tag mounted thereon, or attaching of such a tag to thetool may cause the tool be unwieldy. The inventory of such tools maythus be better suited to visual-scanning methodologies even in systems300 capable of RF-based sensing.

In order to address the deficiencies of the scanning methodologies whenused individually, the tool storage system 300 advantageously usesmultiple scanning methodologies in combination. For example, the toolstorage system 300 may perform an inventory of objects using multiplesensing technologies in accordance with the method 500 of FIG. 5.

FIG. 5 is a flow diagram showing steps of a method 500 for performing aninventory of objects using multiple sensing technologies. The method 500begins at step 501 with the initiation of an inventory scan. Theinventory scan can be manually initiated by a local user (e.g., inresponse to a user request) or remote server (e.g., an asset managementserver), or automatically initiated based on one or more triggers. Forexample, the scan can be initiated in response to a user logging into orout of the tool storage system 300 (e.g., via access control device306), access to the tool storage system 300 being detected (e.g., adrawer 330 or door opening, or closing), on a periodic basis, or thelike.

In response to the scan being initiated, the tool storage system 300 mayfirstly perform a first inventory scan using a first sensing subsystemin step 503. For example, the system 300 may perform an image-based scanto obtain a quick (e.g., near instantaneous) determination of whetherany tools are missing from the tool storage system 300 based on theimage-based scan alone. Once the first scan is completed, the system mayperform a second scan using a second sensing subsystem in step 505. Inturn, in step 507, the tool storage system 300 determines the inventoryconditions based on the sensing data obtained from the first and secondscan results.

While FIG. 5 shows the scanning steps 503 and 505 as being performedsequentially, the scanning steps 503 and 505 can be performedsimultaneously. However, in situations in which the scanning steps 503and 505 are performed sequentially, results from the first scanning step503 can advantageously be used to improve the efficiency or accuracy ofthe second scanning step 505. In such examples, inventory conditions canbe preliminarily determined based on the results of the first scanningstep 503 only, for example prior to performing the second scanning step505. In turn, in step 507, the preliminary inventory conditiondetermination is updated based on the results of the second scanningstep 505.

For example, in an example in which the second sensing subsystem is anRF-based sensing subsystem, a result of the first inventory scan canadvantageously be used to determine how many RF-based tags are expectedto be in the tool storage system 300 and thereby improve the speed atwhich the second scanning step 505 is performed. For example, in a toolstorage system 300 that usually stores ‘m’ tools having associated RFtags, the results of the first inventory scan are used to determine that‘n’ tools having associated RF tags are missing from the tool storagesystem 300. The first inventory scan is then used to determine that the‘m-n’ RF-based tags should be searched for using the second inventoryscan (e.g., an RF-based scan).

In turn, the second inventory scan (e.g., an RF-based scan) is performeda single time (in step 505), and only needs to be repeated if less than‘m-n’ RF-based tags are detected by the first iteration of the secondinventory scan (e.g., the RF-based scan). Thus, the second inventoryscan can be completed very efficiently—notably in situations in whichonly one or a few secondary scans are needed to detect all of the ‘m-n’RF-based tags that are expected to be detected in the tool storagesystem 300.

Finally, an inventory cross-check is performed between the results ofthe first and second inventory scans in step 507 to ensure that theresults of the two scans are consistent. Specifically, the inventorycross-check is performed to ensure that both inventory scans haveidentified the same tools as being present in the tool storage system300 and have identified the same tools as being absent from the toolstorage system 300. User alerts are issued if the results of the twoinventory scans are not consistent with each other.

The previous example has focused on performing an RF-based scan of thetool storage system 300 following an image-based scan. In anotherexample, a bar-code based scan of the tool storage system 300 may beperformed following the image-based scan. Specifically, following theimage-based scan, the result of the first inventory scan canadvantageously be used to determine how many bar-code tags are expectedto be in the tool storage system 300 and the approximate locations ofthe bar-code tags. Specifically, based on the result of the firstinventory scan, the data processing system 140 may retrieve from adatabase stored in memory 144 data on which objects have associatedbar-codes (among the objects identified by the first inventory scan asbeing present), and on the approximate locations within the drawers 330of the storage locations for objects having associated bar-codes. Thebar-code based scan can then be focused on the particular locations atwhich bar-codes are expected to be present in the drawers 330, tothereby increase the efficiency of the bar-code based scan.

In turn, the second inventory scan (e.g., a bar-code based scan) isperformed in accordance with step 505. The second inventory scan can beperformed more efficiently by focusing the scan on the approximatelocations within the drawers 330 of the storage locations for objectshaving associated bar-codes and having been identified as present by thefirst inventory scan. Finally, the inventory cross-check is performedbetween the results of the first and second inventory scans in step 507to ensure that the results of the two scans are consistent. Theinventory cross-check can ensure that both inventory scans haveidentified the same tools as being present in the tool storage system300 and have identified the same tools as being absent from the toolstorage system 300. As above, user alerts are issued if the results ofthe two inventory scans are not consistent with each other.

In the example involving a first image-based inventory scan and a secondbar-code based inventory scan, each inventory scan can be performedusing a respective set of sensor(s). For example, the image-basedinventory scan can be performed using one or more cameras, while thebar-code based inventory scan can be performed using one or morebar-code scanners. However, in some embodiments, both the image-basedinventory scan and the bar-code based inventory scan can be performedusing the same sensor(s), such as the same set of cameras. In suchembodiments, the bar-code based inventory scan can involve locatingbar-codes within images captured by the cameras, reading the locatedbar-codes, and identifying the tool or object associated with each readbar-code in the database. The bar-code based inventory scan can beperformed on the basis of the same images that are used to perform theimage-based inventory scan (e.g., images that are captured in step 503,such as images captured by the cameras 310 while a drawer 330 isclosing), or on the basis of a set of images different from the imagesused to perform the image-based inventory scan (e.g., such as imagesthat are captured in step 505 following the completion of step 503, orsuch as a second set of images captured during step 503). Similarly, incases in which the first and second inventory scans are performed usingdifferent sets of sensors (e.g., cameras for an image-based scan, andbar-code scanners for a bar-code based scan), both sets of sensors mayperform scans of the drawer(s) at the same time (e.g., both during step503) or, alternatively, the different sets of sensors may perform scansof the drawer(s) at different respective times (e.g., one during step503, and the other subsequently during step 505).

As noted above, the RF-based scan, bar-code based scan, or othersecondary scan (e.g., tag-based scan) can be used to identify whether aspecific tool (from among multiple similar tools) has been checked outor checked back in to the tool storage system 300. The RF-based andbar-code based scans can thus be used to determine how many times aparticular tool has been checked out, and/or for how long a duration theparticular tool has been checked out. The tool storage system 300 canthus determine whether the particular tool should be scheduled for are-calibration or other upkeep, for example. In one example, the toolstorage system 300 can thus individually track the usage of differenttorque wrenches and ensure that each torque wrench is recalibrated aftera certain number of uses.

The inventory performed by the tool storage system 300 using multiplesensing technologies can be used to identify the individual user whoreceived and or returned the object/tool, identify the object/tool whichis being issued or returned, place a time stamp on each transactionwithin the system, and store item and user data in a database.

The detailed examples outlined above have focused for illustrativepurposes on embodiments using image-based, RF-based, and bar-code basedsensing technologies. However, the automated asset management system 100can use other combinations of multiple sensing technologies.

For example, the sensing technologies and sensing devices used in thetool storage system 300 can include one or more of opticalidentification sensors, RF identification sensors, direct electronicconnections to tools, weight sensors, contact switches or sensors, sonicemitter/detector pairs, magnetic induction sensing, or the like. Opticalidentification sensors can include sensors for detecting one dimensionalbar-codes with line scanner or camera; sensors for detecting twodimensional bar-codes with camera or other imaging sensor; machinevision identification sensors with camera or other imaging sensor (usingvarious sensing approaches, including UV, infrared (IR), visible light,or the like); and laser scanning. RF identification sensors can includeRFID tags affixed to or embedded in tools (including active RFID tagsand/or passive RFID tags); other RF technologies used in similarcapacity, such as Ruby, Zigbee, WiFi, NFC, Bluetooth, Bluetooth lowerenergy (BLE), or the like. Direct electronic connection to tool caninclude tools that have attached or embedded connectors that plug intoan identification system. Weight sensors can include scales to detectweight of individual objects or of groups of objects; and/or multiplescales used to detect weight distribution within a drawer 330 or otherstorage location or group of storage locations. Contact switches orsensors can include single go/no-go sensors, and/or arrays of sensors todetect shapes or outlines of objects. Sonic emitter/detector pairs caninclude pairings in which an emitter is mounted to the system 100 and adetector is mounted to an object, or an emitter is mounted to an objectand a detector is mounted to the system 100. Magnetic induction sensingcan be used, for example, to locate ferrous tools or products.

As detailed above, the access control device 306 authenticates a user'sauthorization for accessing storage system 300. The methods and systemsused to electronically identify the user requesting access may includeany one or more of the following technologies, and others not mentioned,individually or in combination: RFID proximity sensors with cards;magstripe cards and scanners; bar-code cards and scanners; common accesscards and readers; biometric sensor ID systems, including facialrecognition, fingerprint recognition, handwriting analysis, irisrecognition, retinal scan, vein matching, voice analysis, and/ormultimodal biometric systems.

A detailed example of one illustrative embodiment is provided below. Inthe illustrative embodiment, a physically defined, secure threedimensional object storage device is provided. The storage device is thecontainer from which tools and/or objects are issued and/or returned.The physically defined, secure three dimensional object storage deviceis equipped with a processor and software operative to cause the deviceto communicate electronically directly or through a network with sensingdevices and to obtain data from sensing devices indicating the presenceor absence data of objects within the three dimensional object storagedevice. In the example, the sensing devices used within the threedimensional object storage device include machine vision identificationdevices such as cameras and RFID antennas and decoder.

The physically defined, secure three dimensional object storage deviceis equipped with an electronically controlled locking mechanism, alongwith an access control device including a processor and software meansto electronically identify a user requesting access to the secure areaor object storage device. The processor and software identifypredetermined authorized access levels to the system, and allow or denyphysical access by the user to the three dimensional space or objectstorage devices based on those predetermined authorized levels ofaccess. The access control device used to electronically identify theuser requesting access uses RFID proximity sensors with cards.

The physically defined, secure object storage device is equipped withdrawers. At least one RFID antenna is attached inside the storage deviceand is configured for scanning for RFID tags within the storage device.In embodiments with multiple RFID antennas, different RFID antennas maybe distributed throughout the storage device. The processor and memorystoring executable software program instructions of the storage devicecan be connected to a computer network, and can exchange data with anadministrative software application (e.g., one executed on a remoteserver) used to manipulate and store data and store and displayinformation relative to the data to system users.

In operation, a user scans or approaches an access card to the accesscontrol device of the storage device. The processor of the accesscontrol device (e.g., 306) determines an access level of the user basedon the access card. If the user is determined to be authorized foraccess to the storage device, the authorized user gains access to theobject storage device. In turn, the sensing subsystems (e.g., 150) anddata processing system (e.g., 140) of the storage device are activated.Light emitting diodes (LEDs) used for providing light to the system areactivated, and cameras are activated. In turn, the latch of the storagesystem is unlocked, and the user opens one or more drawers (e.g., 330)and removes or returns one or more objects.

Note that if the user opens an imaging-only drawer (i.e., a drawer whoseinventory condition is determined using imaging only, and not usingRFID), then the RFID scanning subsystem need not be activated and thesystem can use only imaging data. Specifically, the imaging subsystem isused to optionally image the drawer as it opens and to image the draweras it is closed (or once it is closed), and object presence and absenceis determined using only the captured images.

However, if the user opens a drawer for which RFID scanning is used todetermine inventory conditions, a camera-based scan of the drawer isoptionally performed prior to or as the drawer opens. Additionally, theRFID sensing subsystem is activated and an RFID scan may be completedprior to opening the drawer to identify all RFID tags present in thestorage system (or all RFID tags present in the drawer being opened).Specifically, an RFID scan is optionally performed prior to opening ofthe drawer. Additionally, a camera-based scan of the drawer is performedas the drawer closes. In response to the drawer being fully closed, orin response to the user logging out of the storage system, an RFID scanof the drawer or box is performed. The imaging subsystem thus determinesand reports object presence and absence in the drawer, and the RFIDsubsystem scan confirms presence and absence of the specific objects inthe drawer or box using the RFID tag data. Thus, imaging data and RFIDtag data are combined to report presence and absence of all scannedtools, plus presence or absence of serialized items through use of RFIDdata. The inventory scan results are depicted on a display (e.g., 105).As the user logs out, object status is transmitted via network to aprimary database and/or to an administrative application. LED lights areturned off, the lock is engaged, and cameras are set in idle state.

Additionally, the storage system can perform other actions. For example,the system can activate or initiate an RFID scan on the contents of theobject storage device on a scheduled or timed basis between useraccesses and thereby confirm that the contents of the storage devicehave not changed since the last user access.

Unless otherwise stated, all measurements, values, ratings, positions,magnitudes, sizes, and other specifications that are set forth in thisspecification, including in the claims that follow, are approximate, notexact. They are intended to have a reasonable range that is consistentwith the functions to which they relate and with what is customary inthe art to which they pertain.

The scope of protection is limited solely by the claims that now follow.That scope is intended and should be interpreted to be as broad as isconsistent with the ordinary meaning of the language that is used in theclaims when interpreted in light of this specification and theprosecution history that follows and to encompass all structural andfunctional equivalents. Notwithstanding, none of the claims are intendedto embrace subject matter that fails to satisfy the requirement ofSections 101, 102, or 103 of the Patent Act, nor should they beinterpreted in such a way. Any unintended embracement of such subjectmatter is hereby disclaimed.

Except as stated immediately above, nothing that has been stated orillustrated is intended or should be interpreted to cause a dedicationof any component, step, feature, object, benefit, advantage, orequivalent to the public, regardless of whether it is or is not recitedin the claims.

It will be understood that the terms and expressions used herein havethe ordinary meaning as is accorded to such terms and expressions withrespect to their corresponding respective areas of inquiry and studyexcept where specific meanings have otherwise been set forth herein.Relational terms such as first and second and the like may be usedsolely to distinguish one entity or action from another withoutnecessarily requiring or implying any actual such relationship or orderbetween such entities or actions. The terms “comprises,” “comprising,”or any other variation thereof, are intended to cover a non-exclusiveinclusion, such that a process, method, article, or apparatus thatcomprises a list of elements does not include only those elements butmay include other elements not expressly listed or inherent to suchprocess, method, article, or apparatus. An element proceeded by “a” or“an” does not, without further constraints, preclude the existence ofadditional identical elements in the process, method, article, orapparatus that comprises the element.

The Abstract of the Disclosure is provided to allow the reader toquickly ascertain the nature of the technical disclosure. It issubmitted with the understanding that it will not be used to interpretor limit the scope or meaning of the claims. In addition, in theforegoing Detailed Description, it can be seen that various features aregrouped together in various embodiments for the purpose of streamliningthe disclosure. This method of disclosure is not to be interpreted asreflecting an intention that the claimed embodiments require morefeatures than are expressly recited in each claim. Rather, as thefollowing claims reflect, inventive subject matter lies in less than allfeatures of a single disclosed embodiment. Thus the following claims arehereby incorporated into the Detailed Description, with each claimstanding on its own as a separately claimed subject matter.

While the foregoing has described what are considered to be the bestmode and/or other examples, it is understood that various modificationsmay be made therein and that the subject matter disclosed herein may beimplemented in various forms and examples, and that the teachings may beapplied in numerous applications, only some of which have been describedherein. It is intended by the following claims to claim any and allapplications, modifications and variations that fall within the truescope of the present teachings.

What is claimed is:
 1. A method comprising: performing, using animage-based sensing subsystem, an image-based scan of a plurality ofstorage locations for storing objects in an automated asset managementsystem having the image-based sensing subsystem configured to sensepresence or absence of the objects in the plurality of storagelocations; determining, based on the result of the image-based scan,that an object from among multiple similar objects was removed in theplurality of storage locations; performing, using a radio frequency(RF)-based sensing subsystem, an RF-based scan of a plurality of storagelocations for storing objects in an automated asset management system;determining, based on the result of the RF-based scan, that a specificobject from among the multiple similar objects was returned in theplurality of storage locations; and tracking a number of times thespecific object has been checked out based on correlating thedetermination from the image-based scan that an object was removed withthe determination from the RF-based scan that the specific object wasreturned.
 2. The method of claim 1, further comprising tracking, basedon the sensed presence or absence of the specific object in theplurality of storage locations, a total duration during which thespecific object has been checked out from the automated asset managementsystem, and determining whether the specific object requires upkeep tobe scheduled based on 1) the tracked number of times the specific objecthas been checked out and 2) the tracked total duration during which thespecific object has been checked out from the automated asset managementsystem.
 3. The method of claim 1, wherein the specific object is atorque wrench, and the upkeep to be schedule for the torque wrenchcomprises calibration.
 4. The method of claim 1, wherein the performingthe RF-based scan comprises reading unique identifiers stored in RFidentification (RFID) tags associated with objects present in storagelocations of the asset management system.
 5. The method of claim 1,further comprising: performing, using a first sensing subsystem, a firstscan of the plurality of storage locations, wherein the first sensingsubsystem and the RF-based sensing subsystem are configured to eachsense the presence or absence of a same object in the asset managementsystem using different respective first and second sensing modalities,and wherein the RF-based sensing subsystem is operative to differentiatebetween objects that are not differentiated by the first sensingsubsystem.
 6. The method of claim 5, wherein the presence or absence ofobjects in the plurality of storage locations is determined based onresults of both the first scan and the RF-based scan performed using thedifferent first and second sensing modalities.
 7. The method of claim 5,wherein the first sensing subsystem is a camera-based sensing subsystem,and the RF-based sensing subsystem is an RF identification (RFID)-basedsensing subsystem.
 8. An automated asset management system comprising: aplurality of storage locations for storing objects; a radio frequency(RF)-based sensing subsystem configured to sense presence or absence ofthe objects in the plurality of storage locations of the assetmanagement system; a processor; and a non-transitory machine readablerecording medium storing program instructions which, when executed bythe processor, cause the processor to: perform, in response to a user'saccess, using an RF-based sensing subsystem, a first RF-based scan ofthe plurality of storage locations for storing objects in the automatedasset management system; perform based on a pre-determined schedule,using the RF-based sensing subsystem, a second RF-based scan of theplurality of storage locations determine, based on the results of thefirst and second RF-based scans, whether the results of the first andsecond RF-based scans for a specific object from among multiple similarobjects match in the plurality of storage locations; and issue an alertto a user, when a mismatch of the results is determined, with thespecific object from among multiple similar objects the plurality ofstorage locations.
 9. The automated asset management system of claim 8,wherein the processor tracks a number of times the specific object hasbeen checked out, and the processor determines that the specific objectrequires upkeep to be schedules based on 1) the tracked number of timesthe specific object has been checked out and 2) the tracked totalduration during which the specific object has been checked out from theautomated asset management system.
 10. The automated asset managementsystem of claim 8, wherein the specific object is a torque wrench, andthe upkeep for the torque wrench comprises calibration.
 11. Theautomated asset management system of claim 8, wherein the processorperforms the RF-based scan by reading unique identifiers stored in RFidentification (RFID) tags associated with objects present in storagelocations of the asset management system.
 12. The automated assetmanagement system of claim 8, further comprising: a first sensingsubsystem distinct from the RF-based sensing subsystem, wherein thefirst sensing subsystem and the RF-based sensing subsystem areconfigured to each sense the presence or absence of a same object in theasset management system using different respective first and secondsensing modalities, wherein the RF-based sensing subsystem is operativeto differentiate between objects that are not differentiated by thefirst sensing subsystem, and wherein the processor is further configuredto perform, using the first sensing subsystem, a first scan of theplurality of storage locations.
 13. The automated asset managementsystem of claim 12, wherein the presence or absence of objects in theplurality of storage locations is determined based on results of boththe first scan and the RF-based scan performed using the different firstand second sensing modalities.
 14. The automated asset management systemof claim 12, wherein the first sensing subsystem is a camera-basedsensing subsystem, and the RF-based sensing subsystem is an RFidentification (RFID)-based sensing subsystem.